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REMARKS 

The present response is intended to be folly responsive to all points of objection 
and/or rejection raised by the Examiner and is believed to place the application in condition 
for allowance. Applicants assert that the present invention is new, non-obvious and useful 
Prompt consideration and allowance of the claims is respectfully requested. 

Status of Claims 

Claims 1-12, 21 and 24-32 are pending in this application and have been rejected. 

Claims 1 and 21 have been amended herein. Applicants state that no new matter has 
been added by these amendments and new claims. 

CLAIM REJECTIONS 

35 U.S.C. § 103 Rejections 

In the Office Action, the Examiner finally rejected claims 1-4, 6-8, 12, 24-27 and 
29-32 under 35 U.S.C. § 103(a), as being unpatentable over the combination of Castango et 
al. (IEEE Vol. 8, No. 5, Sep. 1998, 562-571), Park et al. (U.S. Patent No. 6,535,632) and 
Trew et al. (U.S. Patent No. 6,173,077). Applicants traverse these rejections in view of the 
remarks that follow. 

Applicants argued that the similarity of Trew et al. to the present invention ends 
with the concept of displaced frame difference (DFD). Trew et al. teaches that, before using 
a motion vector in an image segmentation process, check for each pixel whether the DFD 
exceeds the threshold, and if the DFD exceeds that threshold, motion estimation has failed 
and the motion vector for that pixel is to be disregarded. If one of ordinary skill in the art 
were motivated to combine Trew et al. with Castagno and Park, he would disregard in 
Castagno or Park any motion vector for a pixel, where the DFD exceeds the threshold. 
However, according to the present invention, including DFDs as part of the distance 
measurement leads to an elegant balancing in the segmentation process of the physical 
closeness of pixels, the closeness in RGB or other values of the pixels and the similarity of 
the motion vectors allocated with varying degrees of accuracy at the pixels. In contrast, the 
good approach of Trew et al. is to use the DFD to decide whether or not to throw away the 
motion vector. 
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In response to Applicants' arguments, the Examiner stated that Applicants have not 
submitted claims drawn to the limitations that define the operation and apparatus of the 
invention in a manner that distinguishes over the prior art. The Examiner suggested that 
Applicants amend claims to clearly and distinctly define the claimed invention. 

Applicants have herein amended independent claim 1 to explicitly recite the step of 
"calculating a displaced frame difference by applying a motion vector from the segment to 
the pixel" and to clarify that the displaced frame difference forms part of a distance in 
segmentation vector space which also comprises "differences in pixel values between the 
pixel and the segment" and "differences in motion vector values between the pixel and the . 
segment". Support for this amendment can be found throughout the specification as filed. 

It is thus believed to be clear that claim 1 does not include within its scope an 
allegedly obvious combination which merely "includes" a displaced frame difference, 
where that allegedly obvious combination does not calculate a displaced frame difference 
by applying a motion vector from the segment to the pixel and where that allegedly obvious 
combination does not use a distance in segmentation vector space which comprises said 
displaced frame difference alongside differences in pixel values between the pixel and the 
segment; and differences in motion vector values between the pixel and the segment. 

The Examiner also requested that Applicants provide more evidence to support: 
"use" something but not "include" it. In this regard, Applicants note that it is not attempting 
to distinguish in meaning between these words as such but rather is distinguishing between: 

(a) the general case of the method of segmentation including a displaced frame 
difference (DFD) and 

(b) the specific case of the distance in segmentation sp ace including a displaced 
frame difference (DFD). 

The Examiner contends that the general case is obvious because Castagno teaches 
use of motion vectors in a method of segmentation, because Trew teaches the use of DFD in 
the creation of motion vectors; and because, if Castagno were modified to create its motion 
vectors using the DFD of Trew, then the method of segmentation of Castagno as modified 
would "include" the DFD of Trew. (The Examiner's reliance upon Park can be omitted 
from the present reasoning.) 
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To the contrary, Applicants argue that amended independent claim 1 is not met by 
the general case of the method of segmentation including a DFD, as claim 1 requires the 
distance in segmentation space including a DFD. Applicants refer to the below Diagram 1, 
which indicates a method of segmentation that does not use DFD in any sense. It might be 
regarded as a diagram of the Examiner's allegedly obvious combination — before the 
inclusion of the teaching of Trew. 



Take or measure 
Hand V 
coordinates of 
pixel 



Take or measure 
R, G and B values 
of pixel 



Take or measure 
HandV 
components of 
motion vector at 
pixel 



Determine distances in 
segmentation vector 
space between the 
location of the segment 
and the data point 
representing the pixel 



Determine membership of 
segment by distance in 
segmentation space which 
distance comprises: 

coordinate distance 

(H~Uk) 2 + <V-Mv) 2 

R,G,B differences 

0.5(R - p R ) 2 +1.5(G - Mo) 2 + 0.3{B - 

motion vectef-differences 

10(D h -Mdh) 2 +10(Dv-M™) ! 



It is instructive to consider how such a method might be modified to incorporate the 
teaching of Trew. As previously noted, Trew explains that the method includes the step of 
"identifying pixels for which the motion vectors are invalid by comparing the predicted and 
actual pixel values for the next frame. In this context, "comparing the predicted actual pixel 
values for the next frame" corresponds to calculating the DFD, and the only use of the DFD 
is to identify whether or not the motion vectors are valid. At column 4, lines 24-41, Trew 
states that there are two categories of pixels for which the predicted segmentation data will 
be treated as invalid and specifically "the second category are those pixels which are judged 
as having failed motion compensation". The teaching of Trew with regard to DFD's is then 
clear: before using a motion vector in an image segmentation process, check for each pixel 
whether the DFD exceeds the threshold. If the DFD exceeds that threshold, motion 
estimation has failed and the motion vector for that pixel is to be disregarded. 

The inclusion of Trew in the Examiner's allegedly obvious combination can, 
therefore, be illustrated in Diagram 2 as follows: 
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In this Diagram 2, the DFD for each motion vector is compared against a threshold; 
if the DFD is beneath the threshold, the motion vector is used; if the DFD is above the 
threshold, the motion vector is discarded. Thus, it can be seen that the method illustrated in 
Diagram 2 does not fall within the scope of Claim 1, at least for the reason that the distance 
in segmentation space does not include a DFD. 

For completeness, Applicants provide following Diagram 3, which shows a method 
that would fall within the scope of Claim 1: 
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The Examiner also requested that Applicants provide more evidence to support: 
"counter intuitive step of including a DFD in a distance in segmentation space". In this 
regard, Applicants submit that it may be helpful to take a numeric example. Thus, for 
example: 

H is the horizontal spatial coordinate of the pixel; 
V is the vertical spatial coordinate of the pixel; 
R is the red component of the pixel; 
G is the green component of the pixel; 
B is the blue component of the pixel; 

D H is the horizontal component of the motion (or displacement) vector; and 

D v is the vertical component of the motion (or displacement) vector. 
The location of a segment in the segmentation vector space will have coordinates in each of 
these dimensions and may take the value: u H , p. v , ur, ug, M-b, Hdh, Hdv 

The membership of a segment for each pixel is then determined by a distance in 
segmentation vector space from the data point representing the pixel to the location of a 
segment and an example of such a distance might be: 

(H-Hh) 2 + (V- u. v ) 2 

+ 0.5(R - u. R ) 2 +U(G - Ug) 2 + 0.3(B - u B ) 2 
+ 10(D H - w>h) 2 +10(D v -u dv ) 2 
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The scaling factors used in this example are taken from the formula at page 6, lines 1-8 of 
the application as filed. The smaller this distance, the "closer" is a pixel to the centroid of a 
segment and the more likely it is to be regarded as a member of that segment. 

Looking at this formula, it should be recognized that a pixel will tend to be included 
in the membership of a segment if: 

• It is spatially close the segment, that is to say its H and V values are close to the 
corresponding value of the segment. 

• It has pixel values close to those of the segment, that is to say its R, G and B 
values are close to the corresponding values of the segment. 

• It has the same motion vector as the segment, that is to say its D H and D v values 
are close to the corresponding values of the segment. 

This is reasonably intuitive. If two pixels in an image are spatially close, if they have similar 
color values, and if they are moving in the same direction, then they are likely to be 
representing the same object and should indeed form part of the same segment. 

In accordance with the invention as claimed, a further value would be added to the 
distance measure of the example quoted above, which value is the displaced frame 
difference or DFD. An example of a distance measure may then be: 

(H - tt H ) 2 + (V- + 0.5(R - u R ) 2 +1.5(G - n G ) 2 + 0.3(B - p B ) 2 

+ 10(D H - Hdh) 2 + 10(D V - pov) 2 + 2DFD 2 
A DFD value is generally regarded (and this is consistent with the use of DFD in Trew) as a 
by-product of the technique employed for calculating the displacement vector. The DFD 
value will be zero or small if the displacement vector is accurate and will be larger if the 
displacement vector is less accurate. There is no intuitive reason why any two pixels which 
have the same DFD value should be likely to belong to the same segment. It is, on this 
basis, counter-intuitive to include a DFD value in the distance measurement. 

Thus, amended independent claim 1 is now allowable over the combination of 
Castango et al., Park et al. and Trew et al. Claims 2-4, 6-8, 12, 24-27 and 29-32 are 
dependent from amended independent claim 1, and thus include all the limitations of that 
claim, and are allowable as well. Applicants respectfully request that the Examiner 
withdraw the rejections of claims 1-4, 6-8, 12, 24-27 and 29-32. 
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The Examiner also rejected claim 5 under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Castagno, Park, and Trew in view of Aggarwal et al. (U.S. Patent No. 
6,728,706), rejected claims 9-11 under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Castagno, Park, and Trew in view of Price et al. (U.S. Patent No. 5,606,164) 
and rejected claim 28 under 35 U.S.C. 103(a) as being unpatentable over the combination of 
Castagno, Park, and Trew in view of Penn (U.S. Patent No. 5,848,198). 

In response, Applicants point out that all of claims 5, 9-1 1 and 28 are dependent on 
claim 1. As discussed above, claim 1 is patentable over the combination of Castango et al., 
Park et al. and Trew et al None of Aggarwal et aL, Price or Penn solves the deficiencies of 
those references, as discussed above, such that claim 5 is patentable over the combination of 
those references even in view of Aggarwal et al, such that claims 9-1 1 are patentable over 
the combination of those references even in view of Price et al., and such that claim 28 is 
patentable over the combination of those references even in view of Penn. Applicants 
respectfully request that the Examiner withdraw these rejections. 

The Examiner also finally rejected claim 21 under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Castagno, Park, and Min et al. (International 
Conference on Pattern Recognition, Vol. 1, Sep. 3-7, 2000, pp. 644-647). Applicants traverse 
this rejection in view of the remarks that follow. 

The Examiner commented that Min teaches the concept of a curved-surface data set 
such as toroidal surfaces for segmentation. Applicants have previously pointed out that the 
teaching of Min is that where images comprise representations of curved objects then 
algorithms which attempt to segment region of the image (which are necessarily planar) 
may be improved if those planar regions are treated as projections of curved surfaces. 

Applicants have herein amended independent claim 21 to clarify that the 
segmentation vector space has a canvas that is toroidal "such that the location of a segment 
which would otherwise disappear from one edge of the canvas appears as a result of the 
toroidal shape of the canvas at an opposing edge of the canvas". Support for this 
amendment can be found, for example, at page 8, lines 7-1 1 of the application. 

Amended independent claim 21 clarifies that, according to the present invention, a 
segmentation vector space (defined which is the product of the vector space for feature 
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values and the vector space of pixel addresses) has a toroidal canvas in the sense that the 
location of a segment which would otherwise disappear from one edge of the canvas 
appears, as a result of the toroidal shape of the canvas, at an opposing edge of the canvas. 

Applicants respectfully submit that amended claim 21 cannot be construed as 
including within its scope the arrangement of Min where it is merely proposed that regions 
of the image are regarded as representing surfaces, which may in turn be curved. In the 
claimed invention, the multidimensional segmentation space includes not only the locations 
of pixels in the plane of the image but also feature values at the pixel, such as RGB or 
motion vector values. The simple relationship between a planar region of the image that 
might be regarded as depicting a curved surface in a real world object does not, therefore, 
apply in this multidimensional space. Accordingly, amended independent claim 21 is now 
allowable over the combination of Castango et al., Park et al. and Min. 

In view of the foregoing amendments and remarks, the pending claims are 
allowable. Their favorable reconsideration and allowance is respectfully requested. 

Should the Examiner have any question or comment as to the form, content or entry 
of this Amendment, or if there are any further issues yet to be resolved to advance the 
prosecution of this application to issue, the Examiner is requested to contact the 
undersigned at the telephone number below. 

Please charge any fees associated with this paper to deposit account No. 50-3355. 



Dated: May 11 S 20I0 

Pearl Cohen Zedek Latzer, LLP 

1500 Broadway, 12th Floor 
New York, New York, 10036 
Tel: (646)878-0800 
Fax: (646) 878-0801 




Respectfully submitted^ 



